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AB STRACT 

Results of an evaluation of the South Dakota Project 
to Articulate Continuing Education in Nursing (Project SPACE), which 
was the extent to which the three-year project had been a value to 
South Dakota nurses, were presented. According to the Executive 
Committee's assessment, the project increased quantity and quality of 
continuing education programming, which included refresher courses 
and telecc"* rences, and it established a strong network to 
coordinate continuing education. All registered and licensed 
practical nurses in the state were sent a questionnaire to evaluate 
the project, and 1,670 responded (27 percent). Of the respondents , 81 
percent were registered nurses, and 19 percent were licensed 
practical nurses. Fifty-one percent of the registered nurse 
respondents indicated that their continuing education needs were 
being met, while only 41 percent of licensed practical nurses felt 
their needs were being met. Among the reasons for needs not being met 
are the following: inconvenient location of programs, cost of 
workshops, and inability to obtain release time from the employer to 
attend. In addition to nurses, employers of nurses were surveyed to 
obtain their perceptions of the extent to which the project was 
meeting the continuing education needs of nurses. The quality of 
continuing education activities and their impact on health care in 
the state also wore assessed. During the three years, 42 educational 
offerings were offered to 948 registered and 339 licensed practical 
nurses. (SW) 
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SUMMATIVE EVALUATION OF PROJECT SF ACE 



The South Dakota Project to Articulate Continuing 
Education in Nursing (Project SPACE ) has as its purpose to 
plan for and provide an articulated, coordinated statewide 
system of continuing education for nurses in South Dakota. 
South Dakota is a rural state divided into prairie and plain 
by the Missouri River, with mountains on the western border 
cf the state. South Dakota is primarily an agricultural 
state. Federal Indian reservations comprise 10 percent of 
the land area. The state ranks l6th anonr states with 
respect to land area and h3r& amen" states with respect 

to population. 

In lp80 there were approximately 7,25^ nurses in South 
Dakota- U,977 were registered nurses and 2,277 uere 
licensed practical nurses. Continuing education in South 
Da:ota is voluntary nurses are not required to attend 
cor.tinuinfl education activities to renew their license to 
practice nursing in the state. 

Project SPACE is funded by the Department of Health and 
Human Services, Public Health Service. The project is based 
in the Department of Continuing Education, College of Nursing, 
South Dakota State University. Brookings, South Dakota. 
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Structure and Personnel 



Project SPACE is headed by an Executive Committee 
comprised of individuals representor nursinr Pursinp 
orranizations, such as the South Dakota Board of Nursinr, 
the Couth Dakota Licensed Practical Nurses ' Association and 
others, make up some of tho Executive Connittee membership. 
Other nurses represent schools of nursinr in the state, 
nursing service administration and nursinr prectice. The 
Executive Committee functions to provide direction to the 
project and is the policy nakinr body. 

Directly responsible to the Executive Cornittee ere the 
nroject staff. Project staff consist of the Project Director 
who allocates 25 percent of her time to the project in 
addition to uer responsibilities as head of the Department 
of Continuing Education in the College of HursinP. The 
Project Coordinator devotes full-time to the activities of 

Project SPACE. 

Additional project staff include a half-time Purse 
Resource Librarian and a full-time secretery. Eight Ref.ionrl 
Facilitators comprise the remaining project personnel. 

The project director has as her responsibility the 
dministret.ion of the project, includinp fiscal affairs. 
The coordinator functions to acconr lish the project's 
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objectives. The Ilurse Resource Librarian facilitates the 
library activities in meeting continuinc education needs of 
South Dakota nurses. The regional facilitators represent 
eight geographic regions in South Dakota. The regional 
facilitators are based in agencies /organizations throughout 
che state: contractual agreements are established with the 
employing agency to provide release time for the facilitator 
to devote to Project SPACE activity. The facilitators plan 
and implement continuing education in nursing activities in 
their geographic area under the aegis of the project. In 
some regions the facilitator is assisted by an alternate 
facilitator, which allows for a broader base of support in 
the provision of continuing education activities. 

An Advisory Committee provides information to the 
Executive Committee and to the project staff. Advisory 
Committee members represent nurse anesthetists, licensed 
practical nurses, the Health Education Development Cvstem, 
the South Dakota Association of Health Care Facilities, 
office nurses, the Community Health Education System, 
Veterans' Administration hospitals, nursing homes, the South 
Dakota Health Systems Agency, the South Dakota Hospital 
Services, the Department of Health, the South Dakota Nurses' 
Association, Indian people and others. The Advisory 
Committee membership allows for representation of nursing 
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organizations and providers as well as consumers of health 
care ser vices throughout the state of South Dakota. 

Project Ains and Objectives 

Project SPACE is a three year project. Aims ana 
objectives were established for each of the years of the 
project's duration. The airs and ob, Jtives of the project 
can be viewed at three levels as described by the project's 
go als : 

A) Immediate goal: To provide assistance in l) deter- 
mining the present status of continuing education in nursing 
in South Dakota, and 2) determining the continuing e .ucation 
needs of South Dakota nurses. 

* B) Intermediate goal: To give assistance in meeting 
those needs intended to provide participants with knowledge, 
skills and attitudinal change applied to continuing edu- 
cation activities as developed by an articulated, coordinated 
statewide plan. 

C) Ultimate goal: To ultimately reach the long-range 
goal of continuing ducation through the application cf 
knowledge, skills ad change of attitude intended to improve 
patient ^are. 
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Efforts during the final two years of Prelect ^PACE should 

/ 

be directed toward those act.i\'ties whicja'will have the 
broadest* and most noticeable impact. '"The energies of those 
involved in Project SPACE should not be dissipated on 
activities of little lasting value to nurses in the state.' 1 

DESCRIPTION OF THE FVALUATIOK PROCEDURE 

Id this section of the report, the design of the 
summative evaluation of Project SPACE ie delineated. Sum^ 
mative evaluation is that evaluative process which occurs 
at the conclusion of a pr^rrram or project, in contrast with 
formative evaluation, which occurs during the course of a 
program or project. Formative evaluation often provides 
base-line data which may be compared with data generated in 
the summative evaluation process. The summative evaluation 
generally establishes the extent to which a program or 
project has been worthwhile over the course of its existence. 

The primary concein of the evaluator was the extent to 
which Project SPACE had been of value to the nurses in South 
Dakota. Criteria questions were generated by th%* evaluator, 
which when answered, would address the major concern of 
Project SPACE 1 s worth. The criteria questions were as 
f ol lows : 
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1) To what extent have the „ :s and objectives for 
Project SPACE been .chieved since the formative evaluation? 

2) To what extent do nurses in South DaVota perceive 
that Project SPACE is meeting their continuing education 
needs ? 

3) To what extent has the quality of continuing 
education courses offered by Project SPACE improved over 
the duration of the project? 

h) To what extent Lave the continuing education 
activities sponsored by Project SPACE had an impact on the 
quality of health care in South Dakota? 

The data required to respond to these criteria 
questions were established and appropriate data collection 
techniques identified. The final determination of the data 
sources and data collection techniques was made during a two 
day on-site visit to Project SPACE headquarters at South 
Dakota State University in Brookings, South Dakota. The 
evaluator met with the project staff and outlined the data 
collection process necessary to complete the sumntative 
evaluation. Responsibilities were delineated: the evalu- 
ator would design the data collection instruments; the 
project staff would asnist in the data collection process; 
the evaluator would analyze and summarize the data collected, 
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A tiae frame for the evaluation process was established 
The data collection would be conducted in the spring and 
early summer of the project's final year. The summative 
evaluation report would be completed by mid-summer that 
same year. 

Data Sources and Data Collection Techniques 

The criteria questions generated by the evaluator and 
the data collection techniques which responded to tnesc 
criteria questions are as follows; 



Criteria Question 



Data Source Technique 



To what extent have the 
aims and objectives for 
Project SPACE been 
achieved since the 
formative evaluation ? 



Project staff Interview 
Regional 

Facilitators Discussion 
Executive 
Committee 
Advi sory 
Committee 



Discussion 



Questionnaire 



2. To what extent do nurses Nurses 
in South Dakota perceive 



Questionnaire 
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that Project SPACE is 
meeting their continuing 
education needs? 



To what extent has the 
quality of continuing 
education courses offered 
by Project SPACE improved 
over the duration of the 
pro j ect ? 



Pro j ect files 



Review of 
Documentation 



To what extent have the 
cont inui ng education 
activities sponsored W 
Project SPACE had ai 
impact on the quality 
of health cere in 
South Dakota? 



Regional 

Faci lit ators 
Offering 

Par ti c ipant s 
Participants 1 

Supervi sor s 
Nurses 

Administrators 
Other Health 

Care Groups 
Consumers 



Discussion 

Quest i onnaire 

Quest ionnaire 
Quest ionneire 
Ouest i^nnuire 

Interview 
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PRESENTATION OF EVALU-* " T .'E DATA 

It is the purpose of this section of the report to present 
the evaluative data collected by the various means described in 
the previous ection. Each of the criteria questions will be 
listed followed by that data collected which responds to the 
ouestion . 

1. To what extent have the aims and objectives for Project 
SPACE been achieved since the formative evaluation? 

The evaluator interviewed project staff in relation to 
the extent to which the aims and objectives for Project SPACE 
had been achieved during the period of time (2 years) since 
the formative evaluation had been conducted. The staff seemed 
to think that the majority of the aims and objectives aad been 
achieved to a satisfactory degree. 

in addition, the grant application for Phase I. of Project 
SPACE ves reviewed. The grant application described in some 
detail the achievement of the project's aims and objectives. 
All of the objectives of the project had been realized to 
some ext ent . 
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Ihis criteria question was addressed by the SPACE Regional 
Facilitators in group discussion. A schedule of questions was 
designed by the evaluator to facilitate discussion and response 
to the criteria questions. The schedule consisted of six 
questions, three of which were related to this criteria 
question. These three questions were: l) What heve been the 
accomplishments of Project SPACE? 2) Whf else r.hould Project 
SPACE have accomplished during its existence? and 3) To That 
extent has Project SPACE achieved us goals and objectives? 

The facilitators were presented with the questions an^ 
encouraged to arrive at consensus in their response. A recorder 
was appointed who summarized the group's di -.cussion in writing 
for transmittal to the evaluator. 

The facilitators indicated that : the entire project h«s 
achieved its goals quite well. . .80 percent acnievement of 
project goals and objectives had occurred to date." Several 
facilitators expressed concern that they could have accomp- 
lished more continuing education programming in their regions. 
When asked what else Project SPACE should have accoirp 1 * '■ ■ \ 1, 
the facilitators responded, ". . .A statewide resou- 
could have been compiled and distributed earlier, 
orientation of facilitators who c;ie on staff latr 
reject could have been provided. 1 ' 
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The sane criteria question vas addressed by the project's 
Executive Committee through croup discussion. The Executive 
Committee members discussed the three questions in the same 
manner as had the facilitators, The Executive Committee indi- 
cated that the 'Project had been very successful in meeting 
its goals and objectives and judged that it had exceeded all 
expectations and estimated achievement at I25* r The Executive 
Committee indicated that the project had accomplished much 
more than expected, and so did not identify anything else that 
Project SPACE should have accomplished during its existence. 

Both croups identified accomplishments of the project. 
The regional facilitators agreed that, ' SPACE has tuned into 
the actual needs. Workshops that have been developed are 
very sound- the quality is high. SPACE has been a united, 
cohesive effort as opposed to a competitive effort. The 
planning process has been well developed. More nurses are 
attending workshops. Speakers have developed teaching, skills 
in addition to content knowledge. SPACE brought educational 
opportunities close to home for all nurses. Geograph: , ly 
dispersed conferences on a diversity of topics have r- -i lired 
a cooperative effort of the facilitators. They have . ..r.uo 
a core group of peers highly skilled in provision f • xnuiug 
e-ication. The training SPACE provided to the fa.- r- ,ms 
has been a significant contribution to their exp*. I 
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The Executive Committee stated that the project's accomp- 
lishments included ''increased Quantity and quality of contin- 
uing education programming. A strong network has been developed / 
and is functioning to provide continuing education in a coor- 
dinated fashion. SPACE has become very visible and credible 
in the state and is seen as the 'center' of continuing edu- 
cation activities. Continuing education programming has been 
offered outside tie traditional workshop format; i.e., refresher 
courses and teleconference programming backed on neeis assess- 
ment and planning on a regional basis has helped meet the needs 

of mere nurses . ' 

The Advisory Committee addressed this criteria question 
by means of a questionnaire. Fifteen members of the Advisory 
Committee vere sent the questionnaire with a request that it 
be completed and returned as soon as possible. A self-addressed, 
stamped return envelope was provided. Anonymity of response 
-./as assured. Of the 15, 5, or 335, of the Advisory Committee 
members responded. In response to the question, ! To what 
extent has Project SPACE achieved its goals and objective.?' , 
one individual responded that, 'No doubt they have he.- 
areas where continuing education would have been lee) in.' if 
they had not provided programs.' Another indicate.-: ',. ■ 'A 
lot of time and cost has been put into the project. t-t. 
another stated that, 'SPACE , I'm certain, has enh -•-•1 fchat 
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is available in South Dakota, That has to be of great value 
to rural Sou.h Dakota.- One nerter replied, 'It has been very 
successful in meeting its goals and objectives. I'urres in 
the state are veil aware of SPACE and take advantage of it.' 
One respondent indicated that Ue/she did not have a copy of 
tne goals and objectives of the project, and so did not 
respond to the question. 

2. To what extent do nurses in South Dakota perceive that 

Project SPACE is meetirg their continuing education needs? 



A questionnaire was designed by the evaluator to address 
this criteria question. The questionnaire consisted of 12 
items, 3 of which elicited demographic data about the respondent 
3 of which dealth with continuing education attendance, 1 which 
dealt with reasons for attending continuing education activities 
k which asked whether the respondent's continuing education 
needs were being met, and 1 which asked whether the respondent 
experience* . difference in his/her nursing practice as a 

result of participation in continuing education activities. 

The final item asked for any additional comments the respondent 

wished to make* 
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The questionnaire was sent to all of the registered and 
licensed practical nurses in the state. The qv sst ionnai re was 
mailed as part of the Project SPACE Calendar of Events, which 
lists the continuing education activities sponsored by the 
project. The questionnaire was designed so that it could he 
completed and returned to the project office without affecting 
any of the information contained in the calendar of events. 
The survey form could he folded and mailed without insertion 
into an envelope. Postage for return of the survey forms was 
paid by th« project. 

Of the 8,02U surveys mailed, 1,7^0, or 22 percent, were 
returned as undeliver able . A total of 1,670 completed survey 

forms were returned, a return rate of 27 percent. 

Of the respondents, a majority (8x5) were registered 

nurses. Nineteen percent were licensed practical nurses. 

Those respondents who reported residing and being employed 

in urban and rural areas of the state were almost equally 

divided. 

When asked about continuing education attendance in the 
previous two or three years, registered nurses reported 
attending a median of k workshops, and licensed practical 
nurses reported attending a median of 3 workshops in that 
time period. Next asked whether those workshops were SPACE 
sponsored, registered nurses indicated that a median number 
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of 2 workshops were sponsored by the project, while the median 
number of SPACF sponsored workshops reported by licensed 
practical nurses was one. 

Next asked whether their continuing education needs were 
being met, a majority (5lS) of registered nurses answered yes, 
while 38 percent responded with a no. Ten percent of the 
respondents gave an answer other than yes or no. In contrast, 
only hi percent of licensed practical nurses indicated their 
continuing education needs were being met; a majority (59?) 
responded with a no. Fewer than one percent of both groups 
failed to respond to this item on the survey. 

The respondents were asked to explain their answer. Most 
of the registered nurses who answered yes to the question did 
not offer additional comments, but several offered compliments 
such as, "I think South Dakota is doing a better Job" than 
neighboring states, "I have access to many excellent programs, " 
and "I enjoy attending these worKShops. . .you learn so much 
from them. 

The responses from the registered nurses who indicated 
their continuing education needs were not being met were 
categorized as follows: 

1. inconvenient location of programs 

2. cost of workshops 

3. inability of nurse to obtain release time from 
employer to attend 

ERiC 18 
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h. family responsibilities prohibited attendant:. 

5. inconvenient acheauline of workshops, such as 
no workshops in evenings or on weekends 

6. workshops not offered in area of practice needed 
by nurse 

Those nurses who responded with an answer other than yes 
or no generally indicated their continuing education needs, 
were being met "partially" or "almost." Reasons for these 
responses included identification of other content needed for 
specific areas of practice, or requests that academic degree 
program offer courses in certain parts of the state. 

Most licensed practical nurses who responded yes to the 
question about their continuing education needs being met did 
not offer additi. al comments. Those who did said that their 
employers were offering excellent courses which they attended 
while at work. 

The explanations given by licensed practical nurses for 
their no response were similar to those given by registered 
nurses. The two primary reasons given were the location and 
the cost of workshops. 

In response to an item which stated, "A primary R oa1 of 
Project SPACE is t~ make certain that continuing education 
needs of South Dakota nurses are met. How well would you say 
Project SPACE is doing that now?' , the nurse was asked to 



d in 

ERIC 19 



/ 



SUMMATIVE EVALUATION 
Project SPACE 

Page 18 

indicate his/her response by circling the appropriate number 
on a scale of 1 to 10, where 1 was low and 10 was high- The 
median response for reeistered^nurses was 7 and the median 
response for licensed practical nurses was 5- 

In addition to the survey forms mailed to r.gistered and 
licensed practical nurses in South Dakota, participants at a 
SPACE-sponsored workshop on Legal Aspects in Nursing Practice 
also were given the quest ionnaire . The questionnaire was 
included in the packet of handout materials which each parti- 
cipant received upon registration at the workshop. 

At ,he beginning of the workshop the Project SPACE Coor- 
dinator called the attention of the participants to the survey 
form which was included in their information packet. 
The participants were told that the survey was part of an effort 
to evaluate the effectiveness of the project in meeting the 
continuing education needs of South Dakota's nurses. The 
participants were assured that their responses would be anony- 
mous, and were further told that a similar survey wouj d be 
maileJ »Uh the calendar of events which they would receive. 
The participants were encouraged to also complete the survey 
form included with the calendar. 

There were 113 participants in attendance at the legal 
aspects workshop. Of those present, 95, or 8U percent, 
returned a completed questionnaire. 

ERIC 20 
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In response to the question about whether their contin- 
uing education needs were being met, 12 percent of the respon- 
dents answered. X?J- Those individuals who responded no offered 
as explanation comments similar to those made by the respondents 
to the calendar of events survey. 

In addition to nurses, employers of nurses were surveyed 
to obtain their perceptions of the extent to which Project 
SPACE was meeting the continuing Wucation needs of nurses. 
A survey form was mailed to 58 hospital administrators; the 
directions asked the administrator to assist in the evaluation 
of the project by completing and returning the survey. A 
return envelope was provided; the envelope wab addressed to 
the project evaluator. Of the 58 surveys mailed out, 30 vere 
returned, a return rate of 52 percent. 

The hospital administrators were asked the extent to 
which they perceived that continuing education programs were 
available and accessible to nurses. Most indicated that con- 
tinuinr education was available, .although several stated tfifct^ 
much cf the continuing education offered was 'in the more urban 
areas of the state. One indicated that there was too much 
continuing education available, and called for some coordination 
and evaluation of what was being offered. Much of the concern 
about accessibility was centered on the distance involved for 
the nurses to attend available programs. 
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The administrators were asked about bed size for their 
facility. The responses ranged from l6 to over 500 beds. The 
number of registered nurses employed ranged from h to U8l 
(full-time) and from k to l6U (part-time). The number of 
licensed practical nurses employed ranged from 1 to 177 on a 
full-tine basis and from 1 to 75 employed part-time. 

When asked about the amount of support for continuing 
education attendance for the nurses employed in the responding 
agencies, the amount of paid time provided ranged from a low 
of 25 percent to a high of 100 percent. Twenty-one of the 
30 respondents indicated that all of the nurses' time spent 
in continuing education attendance was paid by zhe agency. 
One respondent, indicated that the amount of paid time provided 
depended on whether the nurse was employed on a full-time or 
part-time basis. 

Estimates of the money provided by the agency for regis- 
tration fees, travel and other continuing education costs also 
ranged from a low of 25 percent to a high of 100 percent. Fewer 
respondents, 18 of the 30, indicated that the facility paid all 
of these costs of continuing education attendance for the nurses 

When asked about the amount of continuing education 
attended by nurses employed in their agency, most adminis- 
trators reported that about half was provided by the agency 
and half was provided by sponsors outside of the employing 
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agency. In one instance, a respondent indicated that all of 
the continuing education attended by nurses in. that facility 
was sponsored by the employing agency, and in one instance the 
reverse was true. In that instance, all of the continuing 
education attended by nurses employed in that particular agency 
was provided by sponsors outside of the employment setting. 

In a final assessment of the extent to which Project SPACE 
had met the continuing education needs of nurses in South 
Dakota, a list of SPACE sponsored continuing education programs 
was compiled. In the three years of its existence, SPACE re- 
ported having sponsored k2 educational offerings. There were 
9U8 registered nurses and 339 licensed practical nurses in 
attendance at those continuing education programs, for a 
total of 1287 participants. 

The attendance figures revealed a marked trend toward 
decreasing numbers of attenders per offering over the three 
year period. .In 1979, for example, there were, on the average, 
38 attenders per offerings. That figure decreased tc 32 
attenders per offerings in 1980, and to 22 in 198l. 

The percentage of registered nurses in comparison with 
licensed practical nurses who attended SPACE sponsored programs 
remained relatively stable over the three year period. For 
example, registered nurse attendance in 1979 was 73 percent 
and licensed practical nurse attendance was 27 percent. In 
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1980 the percentages were 75 Tor registered nurses and 25 for 
licensed practical nurses. In 198l there was 71 percent atten- 

% dance of registered nurses and 21 percent attendance of licensed 

\ 

practical nurses. 

3. To what extent has the quality of continuing education 
activities sponsored by Project SPACE improved over the 
duration of the project? 

In order to* address this criteria question, a review of 
the quality of the educational offerings sponsored by Project 
SPACE was conducted. The quality of the educational offerings 
sponsored by SPACE in .ne beginning of its operations was 
compared with the quality of the educational offerings spon- 
sored by SPACE in the later years of its existence. 

Educational offerings were selected from the Project 
SPACE files from five of the eight regions. One of the edu- 
cational offerings had been held either in 1978 or 1979 (the 
early years of the project) ana anoth-r had been offered either 
in 1980 or 198l (the later years of the project). All of the 
offerings selected had been reviewed by the South Dakota State 
nurses' Association's Continuing Education Approval and Recog- 
nition Program ( CF.ARP ) . All of the offerings had been approved 
by the CEARP committee. This approval indicated that the 
educational offerings were quality educational activities 
for nurses. 
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Each of these educational oITerings was rated on a 
weighted rating scale designed for use in assessing the 
quality of continuing education activities for nurses. ±he 
scale rates the extent to which: 

1. the objectives for the offering are relevant to 
current nursing practice and are based on assess- 
ment of the learners 1 needs 

2. the objectives are measurable (vritten in behavi- 
oral terms) and are attainable in the time allotted 

3. the content is related to the objectives and 
planned in logical c;der 

k. the method involves active participation of the 
learners and is appropriate for the level of the 
learner 

5. the evaluation is related to the objectives 

6. the evaluation is a planned part of the offering, 
and there is a plan for using the evaluation results 

?. the faculty for the offering have an adequate edu- 
cational/experiential background related to the 
topi c 

8. the runners are knowledgeable about nursing, and 
adult education and represent the intended audi- 
ence, ar 

9. the facilities are appropriate to achieve the 
objectives of the offering 

tKfc 25 
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The rating scale had been tested twice for reliability, 
ueing different statistical testing methods. In each instance 
the reliability of the instrument was established at +.80 or 
above for each of the categories, e.g. objectives, content, 

method, and so on. 

An independent rater was obtained to rato the ten edu- 
cational offerings sponsored by Project SPACE which had been 
selected for this study. The rater was selected on the basis 
of demonstrated expertise in credentialing through her past 
experience of at least seven years duration with the review 
and approval process for continuing education in nursing 
activities. The -rater was sent the ten educational offering 
applications and rating forms, with instructions to base the 
educational offering ratings on the criteria established for 
quality continuing education in nursing activities. The 
applications were coded with a number for identification 
purposes; the match between the offerings in later years with 
those of earlier years was not described for the ^ater. 

The results of the rating process, when earlier educat- 
ional offerings were compared with later ones, revealed that 
substantial differences in quality had occurred. Scores on 
the rating forms differed by as many as ten points. 

l». To what extent have the continuing education activities 
sponsored by Project SPACE had an impact on the quality 
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of health care in South Dakota? 
For data to respond to this important criteria question, 
the evaluator surveyed a broad sample of individuals involved 
in health care, in nursing and in continuing education in 
nursing. The data collection methods were varied; among 
those used were discussion, interview, and questionnaires. 

Those individuals involved in Project SPACE were asked 
to give their perceptions of whether health care in the state 
had change over;the p»*t few years. Members of the Executive 
and Advisory Committees indicated, through group discussion 
or in written response on a survey form, that changes had 
occurred and attributed these changes primarily to increased 
specialization of physicians and nurses, increased sophisti- 
cation of the consumers of health care services, technology, 
and the focus of the media on the health care of the citizens 
of the state. 

In group discussion, the regional facilitators indicated 
tnat the changes in health care were due to the development 
of the mec' oal school in the state, the changing role of 
nurses, the increased acuity of patients and the current 
economic situation in the state. All three groups also iden- 
tified the increasing focus on "wellness' , as opposed to 
ilxness, as responsible, in large part, for the change in 
health cere in the state. 
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These three groups were then asked to indicated whether 
any of the changes in nursing practice within the health care 
delivery system could he attrihuted to continuing education. 
The Executive Committee memhers identified an increased aware- 
ness hy nurses of the importance of continuing education »s a 
means for heaping current and competent in their practice. T'ae 
direct relationship between continuing education and change 
in nursing practice was described as "difficult to identify." 

Advisory Committee members offered similar comments. One 
individual indicated that nursing was more sophisticated, but 
could not attribute hat to continuing education alone, since 
nursing was moving toward . reased sophistication along with 
the overall health care delivery system which "is much 
advanced . " 

The regional facilitators agreed with the difficulty in 
obtaining data to support the relationship between continuing 
education and improved nursing practice. They stated they 
thought nurses were more aware of issues and trends in nursing, 
and that changes in practice might be due to the educational 
activity's acting as a catalyst in assisting the nurse with 
initiating a change or in providin- support for a change. A 
change in practice which the regional facilitators attributed 
to continuing education aVtendance was when middle management 
nurses attended workshops in which they learned ways to deal 
more effectively with people. 
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Participants in an educational offering were another group 
from whom data was collected about changes in nursing practice 
due to continuing education attendance. These nurses were 
given a questionnaire at a workshop on coronary care. The 
questionnaire included a listing of the educational offering 
objectives. Participants were asked at the conclusion of the 
offering how we 1 1 they were now able to perform the activities 
described by the objectives, on a scale of 1 to 5, where 1 was 
"not at all" and 5 was "very well." Participants were asked 
to indicate whether they would actually be able to perform 
those behaviors in their work setting and when, and additionally 
were asked to list those factors which would he^v or hinder 
them in performing the behaviors. 

The participant was also given a copy of the questionnaire 
to take to his/her supervisor for completion. The participant 
was told that both questionnaires wouJ d be matched to determine 
whether the individual actually was able to make the change in 
practice which he/su. .nought could be made following contin- 
uing education attendance. The responses of the individuals 
were anonymous, although the two forms were coded so that the 
participants' and the supervisors' forms could be paired. 

Envelopes were provided for the participant's supervisor 
to return the completed form tc the project office. The 
participants' forms were collected at the conclusion of the 
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workshop on coronary care. A to.al of h2 questionnaires were 
distr^D'Ated to the participants at t 1 » workshop-, of those, 37, 
or 88 percent, were returned at the c*>H of the workshop. Only 
3 completed supervisor's questionnaires were returned, a return 
rate of less than 1 percent. 

In one of the three instances where both participant's 
and supervisor's forms were available for review the two 
individuals indicated that the new knowledge could not be put 
into practice. Th* participant was employed in a nursing home, 
and patients with coronary problems were immediately trans- 
ferred to the ne-rest hospital. The supervisor confirmed the 
participant's statements, indicating the participant's "on 
the Job performance rould not allow me to evaluate the 
following. " 

In the second case, the supervisor's rating of the parti- 
cipant's ability to perform the newly acquired skills was 
limited to hearing the employee's report about what he/she had 
learned at the workshop. The supervisor subsequently stated 
that "t-.e employee should be more confident in caring for 
cardiac patients and more knowledgeable due to being updated." 

In the final instance, the supervisor rated the partici- 
pant's ability to translate into practice what had been learne<» 
at the coronary care workshop by reviewing the participant's 
charting. The supervisor rated the nurse's ability to perform 
all of the behaviors described by the offering objectives as 
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"very well" was the highest possible rating. The rating by 
the supervisor took place app. ximately two weeks after the 
date of the workshop. 

Another attempt was made to collect similar data from the 
participants of a workshop, also on coronary care, at a later 
date. The data collection did not occur because the workshop 
was cancelled due to illness of the faculty. 

The general population of nurses also was a^ked to 
address the question of change in practice attributable to 
continuing education participation. A question related to the 
difference, if any, which participation in continuing education 
had made in the nurse's practice was included in the question- 
naire sent to recipients of the project's calendar of events. 
Those r rses who responded to the survey offered examples 
which were categorized in several areas, such as: 

1. increased awareness of the need to keep current 
in practice 

2. increased sense of professionalism 

3. increased feelings of security and competence 
in practice 

li. increased awareness of the need to cont5nue 
education both academically and through non- 
academic means 

5. improved skills, both human relation? and 
procedural 
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Other nurses indicated that participation in continuing 
education helped refresh and reinforce already existing skills. 
Seme indicated that participation in a continuing education 
activity vas an essential first step in returning to nursing 
practice after being out of the field for some time. 

Some of the nurses indicated that continuing education 
had made little, if any, difference in their practice. Many 
indicated that attendance at such programs would only make a 
difference if "they were not so expensive and not so far away." 

Participants in the legal aspects workshop who completed 
the same survey as the calendar recipients gave similar 
responses when asked what difference, if any, they thought 
their participation in continuing education activities had 
made in their subsequent nursing practice. Several indicated 
they ".aught what they had learned in continuing education 
activities to other nurses, through formal in-service class- 
room presentations or informal dialogue with + heir colleagues. 
Several replied, most adamantly, that their participation in 
continuing education was largely responsible for keeping them 
from getting 'burned out" with nursing as had some of their 
peer s . 

Another group of nurses from whom this same data was 
elicited was supervisors in health c-re facilities in the 
state. Survey forms were sent to directors of nursing in the 
58 uospitals and 1*5 nursing homes in the state. Each director 
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was given a numoer of survey forms to distribute to the nursing 
supervisory personnel in that facility with instructions that 
the forms were to be completed and returned to the Project 
S P ACE office in the envelope provided. Responses were anony- 
mous. Responses were received from 38 of the facilities, a 
return rate of 19 percent. 

Responses were received from 21 individuals who indicated 
their position was a director of nursing or comparable position. 
These directors supervised between 0 and 12 registered nurses 
and between 0 and 9 liceased practical nurses. Estimates of 
the amount of continuing education attendance in the previous 
few years by the registerec nurses supervised by the director 
of nursing ranged from none to 100 percent. Estimates of the 
amount of continuing education -attendance by the licensed 
practical nurses also ranged from none to 100 percent. The 
average rate of attendance for registered nurses, however, was 
50 percent and tha average attendance rate for licensed 
practical nurses was nearer 25 percent. j 

When asked for specific changes in nursing practice as / 
a result of continuing education attendance, 6 directors were 
unable to identify any. Most cited as reasons that not enough 
of the nurses in the facility could attend, or that cost and/or 
distance prevented their attendance. One director reaffirmed 
her commitment to continuing education and suggested that 
requiring continuing education attendance for relicensure of 
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nurses would make it easier for him/her to have staff members 
attend continuing education. 

Of those directors who described differences in practice 
as a result of continuing education attendance, several indi- 
cated that new ideas learned in continuing education activities 
were put into practice, although several qualified that response 
by saying "only those new ideas which are applicable to our 
facility." One director identified that nurses were specifi- 
cally implementing techniques learned in a physical assess- 
ment course. One director indicated that nev care plans and 
nursing assessments were the result of nurses 1 attending con- 
tinuing education activities on those topics. Others des- 
cribed such changes as increased awareness of medications 
(actions and reactions) and legal aspects of nursing, improved 
supervisory skills, and more insight into the psycho-social 
needs of patients as resulting from continuing education 
attendance by nurses. 

Nurses who classified vheir position as supervisor or 
comparable also illustrated changes in practice as a result 
of continuing education. There were five individuals who 
indicated they were supervisors; they supervised from one to 
11 other individuals, not all of them nurses. The changes des- 
cribed by the supervisor* were primarily in the areas of legal 
aspects and charting. Several lamented tbs "increased paper 
work" since nurses were documenting their care in more detail. 
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Of the six respondents at the head nurse or comparable 
level, several indicated they were aware of changes in nursing 
practice as a result of nurses 1 participation in continuing 
education. Most attributed those changes to the "new ideas" 
brought back to the facility and implemented. One indicated 
that nurses were better informed as a result of participating 
in continuing education activities and thus were able to give 
better care. 

One staff nurse who responded to the survey indicated 
that the aides in that facility seemed more concerned with the 
1 well-being 11 of the patients after attending a vorkshop on 
geriatrics. Another staff nurse stated that continuing edu- 
cation attendance "keeps things more interesting." 

Hospital administrators were asked to identify changes in 
nursing practice on the part of nurses employed in their faci- 
lities as the result of continuing education attendance. This 
question was an item in the survey which inquired about con- 
tinuing education attendance and reimbursement of nurses 
employed in those facilities. The questionnaire was mailed 
to 58 hospital administrators; 30 responded. 

Of the five respondents who said they had not observed a 
change, one said that was due to being in the position only a 
short while, and another said the "no 1 response was due to 

the inservice programs in the agency covering many of the same 
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areas "on a more practical basis for our hospital*" Two of 

the respondents indicated that there had been "very little" or 

> 

"some" change in nursing practice-attributable to attendance 
at continuing education activities. No explanation for their 
response was offered by those respondents who indicated they 
had not observed changes in practice. 

Those who did observe changes in practice gave examples 
which were classified in the following categories: 

1. increased skills 

2« better attitude toward the profession, the Job 

or continuing education 
3. sharing of information learned, both formally 

and informally 
k. better documentation of patient care; improved 

charting 

5. changes in policies and procedures; ' new ways 
o ! doing old things" 

6. more interest in self- improvement , advanc ^nt 
in the nursing hierarchy 

7. improved communications between various hospital 
departments 

8. increased self-assurance and self-confidence; 
prevention of "burn-out" 



9 

ERIC 



36 



SUMMATIVE EVALUATION 
Project St ACE 

Page 35 

Another population asked to address the question of change 
in nursing practice as a result of attendance at continuing 
education in nursing activities was other health care pro- 
fessions. The SPACE Regional Facilitators were responsible 
for collecting data from health care professionals with whom 
they worked. The facilitators conducted personal interviews 
with -se individuals. / structured interview format was 
used; in each instance the interviewee was told that the 
interview information would be used in the evaluation of the 
effectiveness of Project SPACE. A total of 21 such personal 
interviews were conducted by the regional facilitators. 

Health care professionals interviewed included: physicians, 
respiratory therapists, physical therapists, pharmacists, 
dietitians, medical technologists, social workers and occu- 
pational therapists. Other health care workers interviewed 
hy regional facilitators included nurses aides', a medical 
records technician, a pharmacy technician, a ward clerk, a 
radiologic technologist and a diagnostic ultrasonographer . 

Each of these individuals was asked the extent to which 
health care in the state had changed over the past few years. 
They were then asked to what they attributed the change in 
health care. Most of the responses of these health care 
professionals can be categorized as follows: 

1. increased regulation of the health care industry 

2. increased demand for better prepared workers 
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3. 



increased sophistication of the health care 




S 

high cost of health care services 




6. pvolif ertfcCxuu ol types ueaxou wctr* «wrl;crs 

Other responses addressed the existence of the medical 
school which provided ' more and younger" physicians. Still 
others identified increased consumer awareness and/or the 
trend toward health promotion and illness prevention as 
responsible for the changing health care in South Dakota. 

These same individuals were then asked if they had 
observed a change in nursing practice, and to what they would 
attribute that change. The majority of those who responded 
indicated ths^ the change in nursing practice had been a 
positive one . 

Some of the interviewees responded only vithin the frame- 
work of their own profession, for example, a dietitian des- 
cribed nurses 1 changed behavior related to nutritional needs 
of patients but was not able to identify other areas of change 
in nursing practice. Others made more global assessments of 
change in nursing practice. 

Many indicated they could not attribute the change to 
continuing education. Factors other than continuing education 
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which were thought to contribute +o the change in nursing 
practice included : 

1. the changing role of the nu* 

2. the feminist movement 

3. increased technology 

U. the emphasis on legal aspects which made the nurse 
more conscious of the legal implications of his/her 
actions 

I 5. better basic educational preparation for nursing 

' Those individuals who thought continuing education was 

responsible for the change in nursing practice, at least in 
part, offered specific examples of change they had observed. 
One individual stated that nurses in home health ca* were 
doing more physical assessment of clients than previously. 
Several described examples of nurses' assuming more responsi- 
bility, e.g., giving drugs in a "Code Blue" situation before 
the physician arrived on the scene, making and recording 
nursing diagnoses and so on. Others indicated that nurses 
were taking a more assertive role in patient care, in their 
functioning as members of the health care team, by questioning 
physicians' orders and so on. 

Many of these health care professionals recognized the 
increasingly active involvement of nurses in th^ patient's care 
plan. Several indicated that continuing education partici- 
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pation had made nurses more sure of themselves and their ability 
to contribute to the total health care of the patient. A physi- 
cian attributed the change in nursing practice to continuing 
education because nurses "learn to u» - n«v eauljwotft and • • . 
they review things", and concluded that "continuing education 
probably keeps them better up to date." 

Personal interviews conducted by the facilitators with 
other health care workers produced similar responses. Most 
thought there had been a change in the health care in the state. 
Reasons given to account for the change included increasing 
costs, knowledge about health care on the part of the consumer, 
the acuity of the patients being admitted to health care faci- 
lities, new and expanding health care facilities and the econ- 
omic situation, in bwcn-the state and the nation. Several 
attributed the changes to •prognss.' 

When asked about changes in nursing practice, most of 
these individuals indicated that nursing practice had changed 
as health care had changed. Again some of the reasons for the 
change in health care were not seen as the result of partici- 
pation in continuing education activities but were identified 
as due to tlie expanded roles of nurses, the nursing shortage, 
delegation of many nursing duties to non-professional persons, 
increasing paperwork nurses were ..-quired to do and the increased 
demand for more knowledge and better skills for all health care 
professional*, not Just nurses. 
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Some of the change in nursing practice, however, was 
attributed to nurses' continuing education attendance. Several 
nurses aides identified specific responsibilities they had to 
assume because the nurses were taking on "leadership duties" 
which they learned about in "inservice classes, " One nurses 
aide thought there was "better cooperat ior !T between professionals 
and aides due to continuing education. Yex another individual 
identified that nurses knew more about pharmacology in recent 
years because of having atte~ - continuing education • programs 
on the topic. 

In addition to the personal interviews which they conducted, 
several facilitators distributed the interview forms for com- 
pletion by other members of the health care team in those 
institutions in which they had conducted their personal inter- 
views. These interview forms wer ' then completed and mailed 
to the P^Ject SPACE office. A total of k6 of thebe interview 
forms were received. 

Among those who completed an interview form were: physical 
therapists, a minister, nurses aides, a materials manager, 
housekeepers, a physican's resistant, individuals from Per- 
sonnel Departments, social vorkers t an Emergency Medical Tech- 
nician, and others. One registered nurse also completed the 
interview form. 

le responses from this group were fai *ly similar to those 
of the individuals personally interviewed by the regional faci- 
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litators. Most indicated there had been a charige in health 
care in the state. Again the reasons for the change in health 
care were typical of other health care individuals 1 responses: 
increasing regulation of the industry, higher costs, more 
consumer education and awareness, advances in technology and 



These individuals, for the most part, indicated that there 
also had been a change in the practice of nursing. Many of 
them attributed that change, either wholly or in part, to 
continuing education. Those who did not describe the change 
as due to continuing education offered other reasons for the 
change, such as federal regulation, research, new discoveries 
and the nursing shortage. 

Those individuals whe attributed change in nursing practice 
to continuing education offered examples such as improvemert in 
the nurses' dealing with dying patients and observation of "more 
compassionate responses 1 ' to patients 1 concerns. One individual 
identified more attention to documentation of nursing care on 
the patient's medical record as a positive change in nursing 
care attributable to continuing education attendance, but 
expressed concern that the increased paper work left the nurse 
with less time to provide "TLC" (tender, loving care) for the 
patient, and also possibly increased the change for "burn-out" 
of the nurse. 



so on. 
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One individual noted the "growing sense of prof essionalism" 
among nurses, attributed that to nurses 1 attendance at work- 

shops on nursing issues and trends. Another individual thought 
that nurses' attitudes toward the elderly had changed because 
of the continuing education programs on gerontology. Still 
others related examples of positive changes in nurses 1 atti- 
tudes toward other members of the health care team* A house- 
keeper stated that nurses were now 'Vore friendly/' 

Others reported that cont inu ^ng education had assisted 
nurses to specialize in certain areas of practice as well as 
in assuming expanded roles, such as that of a family cr pedi- 
atric nurse practitioner. One respondent, a physician, indi- 
cated that continuing education had had a detrimental impact 
on the role of the nurse: "we no longer have the dedicated 
nurse." Several nurses aides commented that nurses now were 
"better supervisors . " 

In general, most of those who responded, either to the 
personal interview or to the written interview survey form, 
indicated there had been some positive change in nursing 
practice. Most also attributed some, if not all, of that 
change to nurses' attendance at continuing education activities. 

Another group asked about continuing education of nurses 
in South Dakota was consumers of nursing services. Two regional 
facilitators interviewed consumers in shopping centers in the 
state . The consumers were asked several questions related to 
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health care in tne state. Then they were asked to rate nurses 
and physicians on several characteristics. The six character- 
istics rated included three which were related to continuing 
education: competence, keeping up with what's new and continuing 
their t ucation. The individual interviewed was asked to rate 
these characteristics on a scale of 1 to 10, where 1 was low 
and 10 was high. 

Fifteen interviews were conducted. Of the 15 consumers 
inter-iewed two were housewives, one was a businessman in a 
small tcwn^population: hOO) , one was a farmer and one was a 
teacher, amorg others. 

When rating the characteristic of competence, 9 of the 
coasuuers rated both nurses and physicians the same. Five of 
th' 15 rated physicians higher in competence than nurses, and 
1 of the 15 rated nurses higher than physicians. None of the 
ratings by consumers of competence of physicians and nurses was 
lover than 7 ■ 

The rating of the characteristic, "keeping up with what's 
new," was not lower than 6. Of the 15 consumers, only 3 rated 
nurses and physicians the same. Five rated nurses higher than 
physicians, and 7 rated nurses lower. 

When the characteristic "continuing their education" was 
rated by consumers, 5 of the 15 rated both nurses and physicians 
the same. Two rated nurses higher and 8 rated nurses lower 
than physicians on this characteristic None of the consumers 
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interviewed rated this characteristic less than 5 for either 
physicians or nurses, 

DISCUSSION AND CONCLUSIONS 

This section of the report presents the assessments and 
conclusions drawn by the summati;e evaluetor. The conclusions 
reflect the analysis of the evaluative data collected during 
the summative evaluation process. 

At the time Project SPACE was established, goals, aims, 
and objectives were identified. The formative evaluation of 
the project attempted to measure ^he extent to which those 
objective had been achieved early ir the project's existence. 
Those aims and objectives had not been set in order of priority: 
as priorities emerged during the life of the project, the 
objectives were attained. Clearly some objectives were more 
important to \,he project than were others. It is also clear 
from the data collected during the summative evaluation process 
that those objectives which were significant for the project to 
achieve in order to accomplish its mission have been satis- 
factorily achieved. 

Credit for the achievement of the aims and objectives of 
the project is largely due those who were the most integrally 
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involved in the project's day-to-day activities: the project 
staff and the regional facilitators. The exemplary leadership 
of the project staff is the pivotal element in the project's 
success. The varying level of commitment to Project SPACE on 
the part of the regional facilitators is a natural phenom on 
which is not seen hy the evaluator as having detracted in the 
slightest from the project's accomplishments. 

Credit for the project's having successfully achieved its 
objective, is also due those who were involved, although to a 
lesser extent, in project activities: the Executive Committee 
and the Advisory Committee. Without the support of these groups 
the project would certainly have experienced difficulty in 
achieving its goals. 

One of the major goals for the project was to meet the 
continuing education needs of South Dakota's nurses. This goal 
was an ambitious undertaking, but one which the project attained 
to a credible degree. 

The plethora of data collected to i spond to this concern 
indicates that the project has done a remarkable Job in attemp- 
ting to ascertain nurses' continuing education needs and program 
to meet those identified needs. Project SPACE is not the pri- 
mary provider of continuing education offerings in the state, 
nor is providing continuing education activities one of its 
most significant goals. Rather the project focused its atten- 
tion on coordinating continuing education activities throughout 
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the stcte to minimise duplication and fragmentation and to 
effectively utilize scarce educational resources. 

It is interesting to note that those nurses who were in 
attendance at a continuing education offering were more posi- 
tive about the extent to which Project SPACE was meeting their 
continuing education needs than were their counterparts who 
responded to a by-nail survey. The more positive response by 
registered nurses than by licensed practical nurses can perhaps 
be attributed to the higher continuing education attendance 
rates of registered nurses than licensed practical nurses- 

Reasons given by both registered and licensed practical 
nurses in explaining why their continuing education needs were 
not being met described primarily those factors which can be 
labeled "personal inconvenience.' Continuing education atten- 
dance studies in the nursing and other literature reveal that 
such reasons are more a result of lack of motivation on the 
part of the -Individual than of real personal hardship which 
prohibit s attendance . 

The number of continuing education offerings sponsored 
by Project SPACE over its lifetime reflects a trend toward 
planning and implementing the same number of courses, but 
enrolling fewer participants per course, in other words, 
having smaller classes. It seems unlikely that the number of 
participants could be expected to increase; offerings seem to 
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be close to the demand it th^ voluntary market. No great 
increases would be anticipated until participation in contin- 
uing education becomes required for relicensure of nurses in 
South Dakota. 

It is more di" icult to document the quality of offerings 
than the quantity, but the evidence suggests that in general 
the quality of the continuing education offerings sponsored 
by Project SPACE is high. It is reasonable to assume that the 
consistent submission of SPACE sponsored continuing education 
programs for approval through the State Nurses' Association 
quality assurance mechanism has maintained the high quality 
of the courses. 

It is equally as impossible, or maybe more so, to measure 
the quality of health care in the state. To attempt to quantify 
th^ impact of continuing education on the quality of that health 
care, and in particular, nursing care is an inconceivable task 
in the context of aa evaluation process such as this. ^aere 
is, however, a considerable amount of anecdotal evidence of 
improved health care in the state nf South Dakota that can be 
directly traced to nurses' attendance at continuing education 
offerings . 

The most significant aspect of this conclusion is that 
the e/idence which points to a positive impact on nursing care 
as the result of continuing education attendance is that the 
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data which was collected was not only from nurses in practice, 
and nurses who supervise them, but also fruu. other health care 
workers. I* is fairly common in the impact evaluation liter- 
ature to report self-assessments by those t*ing studied where 
changes in practice are attributed to an educational inter- 
vention. > 

Beyond that, though, the data collected in South Dakota 
indicates that others, physicians, social workers, pharmacists 
and so on, also perceive that nurses 1 practice has changed as 
a result of their participation in continuing education acti- 
vities. It is gratifying that most of the changes are deemed 
to be positive ones. Even the physician's comment that nurses 
are no longer "dedicated" sounds a note of triumph for those 
who have so diligently strived to free themselves from the 
"physician's handmaiden role" to which nurses have been 
relegated for so long. 

That participation in continuing education activities is 
assistive in nurses 1 maintaining currency in practice is echoed 
and re-echoed throughout the data collected in this summative 
evaluation. In addition, continuing education is credited with 
increasing the nurse's confidence as a professional and with 
preventing burn-out. A review of the content of SPACE spon- 
sored continuing education offerings provides soue indication 
that those programs may have been instrumental in achieving 
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the purposes of continuing education beyond updating know- 
ledge and skills of nurses. 

In conclusion, it is apparent through this summative 
evaluation process that the three goals of Project SPACE have 
been achieved. The ultimate goal of the project, the : ' . . . *?pli 
cation of knowledge, skills and change of attitude intended to 
improve patient care ,; has been reached. Not only do nurses 
themselves testify to this. b" + -iiu tneir colleagues, super- 
visors, other members of the health care team, and consumers 
of their services all lend credence to their assertions. That 
Project SPACE can rightly assume it had an essentia? role in 
effecting such an improvement in nursing care in the state of 
South Dakota is an achievement of boundless magnitude. 
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